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Introduction 


The  project  aimed  to  develop  a  battery  of  tests  to  diagnose  visual  dysfunc¬ 
tions  in  cognitively  impaired  observers  based  on  their  oculomotor  reflexes. 
The  oculomotor  reflexes  provide  a  simple  and  robust  method  to  study  vision 
in  passive/immature/impaired  observers.  For  example,  oculomotor  reflexes 
are  widely  used  to  study  vision  in  infants  and  in  marketing  research.  The  pro¬ 
posed  method  is  versatile,  applies  to  a  wide  range  of  visual  mechanisms,  and 
can  provide  both  qualitative  (yes/no)  and  quantitative  (degree)  estimates  of 
the  visual  loss.  The  project  embodies  a  full  research  and  development  cy¬ 
cle,  from  establishing  the  most  effective  stimuli  for  various  kinds  of  visual 
dysfunctions  to  the  design  of  a  computerized  testing  kit  suitable  for  use  by 
non-specialists.  After  a  short  training  course,  the  kit  could  be  used  at  army 
bases  and  local  hospitals  to  make  a  quick  initial  diagnosis. 
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Body 


Title 

▼  1)  Hardware  development 

•  1.1)  Shopping  for  hardware 

•  1.2)  Testing  the  hardware  kit 

•  1.3)  Finalizing  the  hardware  kit 

▼  2)  Software  development 

•  2.1)  Stimulus  presentation  software 

•  2.2)  Data  acquisition  software 

•  2.3)  Data  analysis  software 

•  2.4)  Adaptive  stimulus  presentation  sofware 

•  2.5)  Data  analysis  and  diagnosis  inference 

▼  3)  Visual  tests  development 

•  3.1)  Identify  target  visual  dysfunctions 

•  3.2)  Design  the  corresponding  visual  stimuli 

•  3.3)  Pilot  testing  of  the  stimuli 

•  3.4)  Test  stimuli  for  normal  subjects 

•  3.5)  Test  stimuli  for  subjects  with  vis... 
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Figure  1:  Gantt  chart  for  the  project. 

As  specified  in  the  Statement  of  Work,  the  main  goal  of  the  first  12 
months  of  the  project  was  to  design  effective  stimuli  allowing  the  oculomotor 
reflexes  to  be  used  as  diagnostic  tools  for  various  visual  dysfunctions.  The 
three  reflexes  proposed  for  this  project  (optokinetic  reflex,  pupillary  reflex, 
orienting  reflex)  form  a  complementary  set.  For  example,  our  preliminary 
data  demonstrated  that  optokinetic  reflex  in  response  to  drifting  gratings 
is  an  effective  tool  for  testing  various  low-level  visual  mechanisms,  such  as 
visual  acuity,  contrast  sensitivity,  color,  and  depth  sensitivities.  On  the  other 
hand,  orienting  and  pupillary  responses  to  salient  stimuli  can  be  used  to 
test  higher-order  visual  functions,  such  as  shape  perception,  face  recognition, 
visual  memory,  etc.  Even  if  the  subject’s  eye  movements  are  impaired  by  the 
injury,  the  pupillary  reflex  alone  can  be  used  as  a  qualitative  diagnostic  tool 
for  both  the  low-level  and  the  high-level  visual  functions. 
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The  main  goal  of  the  second  12-months  period  of  the  project  was  to 
make  the  tools  developed  in  the  first  aim  available  to  non-specialists.  To 
this  end,  a  computerized  testing  kit  based  on  an  automatic  eye-event  regis¬ 
tration  algorithm  was  developed.  The  kit  is  used  to  present  visual  stimuli, 
collect  eye-position  and  pupil-size  data  and  search  the  data  for  relevant  ocu¬ 
lar  reflexes  evoked  in  a  real-time  automated  regime.  Based  on  the  analysis, 
an  adaptive  algorithm  chooses/adjusts  the  test  stimuli  for  the  next  round  of 
stimulation  until  a  certain  reliability  level  is  achieved  and  the  battery  of  tests 
is  exhausted. 

The  main  goal  of  the  third  and  final  12-month  period  of  the  project  was 
to  test  a  large  cohort  of  normal  observers  on  the  developed  stimuli  and  hard¬ 
ware  kit  in  order  to  accumulate  the  normative  distribution  of  performance 
scores.  It  was  also  planned  to  test  a  large  number  of  patients  with  visual 
dysfunctions.  Then,  using  the  normative  distribution,  the  test  results  for  the 
observers  with  visual  dysfunctions  in  the  form  of  percent  correct  performance 
scores  could  be  translated  to  the  probabilities  of  various  visual  dysfunctions, 
hence  validating  the  proposed  diagnostic  method. 

Figure  1  shows  a  Gantt  chart  indicating  project  tasks  and  their  cur¬ 
rent  state  of  completion.  The  following  project  goals  were  met:  we  selected 
and  purchased  all  the  necessary  hardware  for  the  project,  identified  com¬ 
mon  visual  dysfunctions,  designed  stimuli  testing  for  these  dysfunctions,  and 
developed  the  necessary  software  to  present  the  stimuli,  capture  the  eye¬ 
tracking  data,  and  analyse  it  for  the  relevant  eye-events.  This  completed 
the  hardware  and  software  kit  building  phase.  Ocular  reflexes  evoked  by  the 
developed  stimuli  were  captured  for  120  normal  observers.  The  data  were 
used  to  calculate  normative  distributions  and  predictive  power  of  each  of  the 
tested  stimuli.  These  tasks  are  described  in  detail  in  the  following  sections. 


Hardware  development 

Different  models  of  eye-trackers  were  investigated  in  the  first  quarter  of 
year  1.  The  goal  was  to  find  the  model  best  suited  for  the  project,  i.e., 
portable,  and  well  suited  for  experimental  work  with  cognitively  impaired 
subjects.  Other  search  parameters  included  programming  interface  capabil¬ 
ities,  ease  and  speed  with  which  hardware  customizations  could  be  made, 
and  the  overall  cost.  After  investigating  several  eye-tracker  models  being 
used  on  Northeastern  University  campus  and  visiting  local  manufacturers 
of  eye-tracking  equipment  we  purchased  two  video-based  eye  trackers  from 
ISCAN  Inc.  (based  locally  in  Woburn,  MA):  one  in  remote  and  one  in  head- 
mounted  configuration  (Figure  2).  This  allowed  us  to  test  both  eye-tracking 
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Figure  2:  Two  ISCAN  Inc.  eye-trackers  used  for  the  project.  The  head-mounted 
unit  is  mounted  on  a  safety  goggles  frame.  The  remote  unit  is  positioned  next  to 
the  monitor. 


modes  and  to  find  the  one  best  suitable  for  this  project. 

The  eye-tracking  units  came  with  a  laptop  and  desktop  computers.  The 
laptop  computer  is  more  portable,  but  currently  can  only  be  used  for  120  Hz 
eye-tracker  sampling  rate.  The  desktop  can  be  used  for  240  Hz  sampling  rate, 
but  is  less  portable.  Accumulated  data  indicates  that  120  Hz  is  adequate  for 
our  purposes,  but  we  are  leaving  the  240  Hz  option  open,  because  the  higher 
sampling  speed  might  be  necessary  for  noisier  data  (subjects  with  droopy 
eyelids,  thick  glasses,  uncontrolled  head-movements,  etc.). 

The  complete  hardware  kit  (the  head-mounted  configuration)  finalized  by 
the  end  of  year  1  is  shown  in  Figure  3.  To  ensure  maximum  performance 
two  computers  (laptops)  are  used.  One  computer  is  dedicated  to  controlling 
the  eye-tracker.  Stimulus  generation,  presentation,  data  collection,  and  data 
analysis  are  carried  out  by  another  computer.  The  two  computers  communi¬ 
cate  via  serial  connection.  Eye  images  are  sampled  at  120  Hz  by  the  ISCAN 
Inc.  eye-tracker.  Video  components  of  the  eye-tracker  include  miniature 
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Figure  3:  The  complete  hardware  kit.  The  data-collection  laptop  is  on  the  left,  the 
video-presentation  and  data  analysis  laptop  is  on  the  right.  The  eye-tracker  is  in 
the  center,  its  electronics  (MiniLab)  is  in  the  back,  the  head-mounted  eye  cameras 
and  IR  mirror  attached  to  a  safety- google  frame  are  in  the  front.  The  Keyspan 
serial-USB  adapter  connecting  the  eye-tracker  to  the  data-analysis  laptop  can  be 
seen  in  the  center. 


eye-  and  scene-cameras,  infrared  (IR)  illuminator  LED,  and  an  IR  mirror  all 
mounted  on  a  safety-goggles  frame  positioned  on  a  subject’s  head.  The  elec¬ 
tronic  components  (MiniLab)  include  hardware-implemented  algorithms  for 
segmenting  the  pupil  image  from  the  eye  image  and  calculating  the  pupil’s 
position  and  size.  MiniLab  also  includes  hardware  for  serial  data  output. 
The  eye-tracker  is  controlled  from  the  laptop  shown  on  the  left.  ISCAN  soft¬ 
ware  installed  on  the  laptop  provides  an  intuitive  graphical  user  interface 
(GUI)  allowing  to  control  the  eye-tracker  in  a  simple  fashion.  In  particular, 
it  is  used  to  optimize  the  camera  settings  for  various  lighting  conditions  and, 
if  required,  for  glasses  or  contact-lenses  worn  by  the  subject.  The  laptop 
shown  on  the  right  is  used  for  visual  stimulation  and  online  analysis  of  the 
eye-tracking  data.  Visual  stimuli  are  presented  either  on  the  laptop  screen 
or  on  a  dedicated  monitor  (not  shown).  The  eye-tracking  data  is  received 
in  real-time  regime  from  the  MiniLab  electronics  via  a  6  ft.  serial  cable. 
The  data  is  received  on  a  USB  port  of  the  laptop  via  a  Keyspan  serial-USB 
adapter  (model  USA-19HS). 

The  head-mounted  version  of  the  eye-tracker  can  be  mounted  on  a  base¬ 
ball  cap  if  the  safety-goggles  frame  is  not  tolerated  by  the  subject.  If  neither 
version  of  the  head-mounted  device  can  be  safely  worn  by  the  subject  the 
remote  (desktop  mounted)  eye-tracker  with  automatic  head  tracking  can  be 
used.  The  remote  eye-tracker  can  also  be  used  for  detecting  dysfunctions  in 
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binocular  convergence  for  all  subjects,  because  it  allows  to  track  both  eyes 
simultaneously  (stimuli  are  not  currently  implemented).  The  hardware  kit 
was  tested  both  in  Pi’s  laboratory  and  in  field  conditions.  In  particular, 
the  kit  was  transported  to  Brown  University,  where  a  subject  with  recent 
TBI-related  visual  dysfunctions  was  tested  in  typical  office  settings.  The  kit 
weighing  about  20  pounds  can  be  transported  in  a  medium-sized  backpack. 
It  takes  approximately  10  minutes  to  assemble  the  kit,  to  power  up  the  laptop 
computers  and  to  get  them  ready  for  the  test. 


Visual  tests  development 


Figure  4:  Examples  of  visual  stimuli  used  for  the  study.  Drifting  gratings  (top 
left)  were  used  to  evoke  optokinetic  reflex,  neutral  and  mildly  offensive  words 
(top  right)  were  used  to  evoke  pupillary  response,  images  of  odd-ball  and  mildly 
arousing  female  figures  were  used  to  evoke  orienting  reflex  (saccades)  and  pupillary 
response. 

In  the  second,  third,  and  forth  quarters  of  year  1  we  collected  and  ana¬ 
lyzed  pilot  data  for  normal  observers.  Pilot  stimuli  included  various  drifting 
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gratings  and  odd-ball  images  of  faces,  bodies,  and  words.  Figure  4  shows 
some  examples.  The  images  were  obtained  from  freely  available  research 
databases.  Typical  pilot  session  lasted  6  minutes  during  which  30  -  40  visual 
stimuli  were  shown.  Subjects  were  viewing  stimuli  on  a  21”  LCD  monitor 
from  a  distance  of  60  -  80  cm.  The  head-mounted  version  of  the  eye-tracker 
was  used  for  all  subjects,  eye-movements  were  sampled  at  120  Hz.  At  this 
stage  the  data  were  collected  off-line:  recorded  on  a  hard  drive  during  the 
experimental  run  and  analyzed  after  its  completion.  The  pilot  data  was  used 
to  optimize  the  visual  stimuli  and  to  design/test  eye-event  Liters  as  described 
below.  A  fragment  of  the  pilot  data  for  one  subject  is  shown  in  Figure  5. 

Eventually,  the  stimuli  were  used  to  evoke  reflex  eye-movements  in  120 
normal  observers.  We  received  clearances  from  the  Northeastern  University 
IRB  and  MEDCOM  USAMRMC  for  work  with  human  subjects  in  March 
2011.  PI  and  his  graduate  students  were  used  as  subjects  prior  to  this  date. 
We  used  inexperienced  observers  naive  to  the  purpose  of  the  test:  observers 
only  knew  that  they  were  taking  part  in  an  eye-tracking  experiment.  The 
only  instructions  received  from  the  experimenter  were  to  attend  to  the  screen. 
Such  experimental  conditions  were  chosen  to  simulate  testing  cognitively 
impaired  observers. 


Reflex 

Stimulus 

Functions 

Visual  Nystagmus 

luminance  grating 

equiluminant  grating 

contrast  gratings  at  3  spatial  scales 

semi-coherently  drifting  dots 

low-level  motion 

color  vision 

visual  acuity 

high-level  motion 

Orienting  Reflex 

contour  /  texture  boundary 

odd  image  (animal  among  males) 

seminude  female  vs.  ’boring’  male 

low-level  object  processing 

high-level  object  processing 

high-level  object  processing 

Pupillary  reflex 

mild  curse  in  a  random  word  stream 

odd  word  within  a  sentence 

mid-level  reading 

high-level  reading 

Table  1:  Oculomotor  reflexes,  the  stimuli  used  to  evoke  the  reflexes,  and  the 
corresponding  tested  visual  functions. 

The  pilot  stimuli  developed  and  tested  during  year  1  were  improved  at 
the  end  of  year  2  to  produce  more  robust  eye  events  suitable  for  automated 
detection.  The  finalized  battery  of  visual  stimuli  along  with  the  associated 
ocular  reflexes,  and  the  corresponding  tested  visual  functions  are  described  in 
Table  1.  In  particular,  we  developed  a  new  stimulus  for  visual  nystagmus  to 


test  visual  motion  processing  in  the  presence  of  incoherent  motion  noise.  The 
drifting  equiluminant  color  grating  stimulus  used  for  testing  color  deficien¬ 
cies  was  redesigned  to  make  the  evoked  visual  nystagmus  much  more  robust. 
Because  the  absolute  gaze  is  not  measured  in  our  paradigm  (this  would  re¬ 
quire  a  gaze  calibration,  involving  significant  cooperation  of  the  subject)  the 
orienting  reflex  has  to  be  measured  with  respect  to  an  undefined  gaze  po¬ 
sition  at  the  time  of  the  stimulus  onset.  To  attract  the  gaze  to  the  center 
of  the  screen  a  disk  contrast-reversing  for  500  msec  was  added  at  the  center 
of  the  screen  1000  msec  before  the  stimulus  onset.  In  addition,  the  odd-ball 
image  and  contour/boundary  stimuli  used  to  evoke  the  orienting  reflex  were 
simplified  to  make  the  evoked  eye  movements  less  dependent  on  the  gaze 
position  at  the  time  of  the  stimulus  onset.  To  this  end,  the  odd  image  or  the 
contour/boundary  was  positioned  in  one  of  the  4  quadrants  on  the  screen 
(compared  to  8  such  positions  used  in  pilot  stimuli).  The  test  was  considered 
passed  if  the  subject’s  gaze  stayed  in  the  ’interesting’  quadrant  significantly 
longer  than  in  the  remaining  quadrants.  The  odd-word  stimulus  originally 
designed  to  test  the  ability  to  read  and  comprehend  isolated  words  was  ex¬ 
tended  to  include  complete  sentences,  where  the  ability  to  understand  the 
logical  structure  of  a  written  text  is  also  tested. 


Software  development 

Once  the  eye-tracking  units  arrived  in  January  2011,  we  started  software 
development  for  visual  stimulus  presentation,  eye-tracking  data  readout,  and 
the  data  analysis. 

Visual  stimulus  presentation  and  data  import 

A  software  library  for  adaptive  visual  stimulus  generation,  presentation,  and 
analysis  (C++  and  OpenGL  based)  was  developed.  This  library  is  used  to 
create  visual  stimuli  and  to  collect  real-time  data  from  the  eye-tracker.  The 
data  is  received  from  the  eye-tracker  computer  via  serial  connection.  The 
library  incorporates  a  graphical  user  interface,  which  allows  to  easily  observe 
eye  events  and  to  vary  the  stimulus  parameters. 

Eye-event  filters 

Using  the  collected  pilot  data  we  designed  software  filters  for  automatic  de¬ 
tection  of  the  relevant  eye  events  in  real  time.  Altogether,  4  types  of  filters 
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were  developed:  blinks,  saccades,  optokinetic  reflex  (nystagmus),  and  pupil¬ 
lary  response  (contraction/dilation).  The  filters  are  based  on  measuring  and 
thresholding  eye-movement  velocities  and  pupil  area  changes.  The  filters 
were  designed  to  be  used  with  real-time  data:  they  are  fast  and  they  do  not 
require  large  amounts  of  data  to  work.  About  one  second  of  data  is  suffi¬ 
cient  to  detect  saccades  and  pupillary  reflex,  and  several  seconds  of  data  -  to 
detect  optokinetic  reflex  (which,  typically,  lasts  for  3  -  5  seconds  at  a  time). 

In  the  pilot  stage  (year  1)  the  filters  were  validated  using  human-parsed 
fragments  of  the  pilot  data.  The  collected  eye-tracking  data  were  searched 
and  parsed  into  different  types  of  relevant  eye  events,  such  as  blinks,  visual 
nystagmus,  saccades,  and  pupil  dilations.  Half  of  the  data  were  used  to  design 
the  software  filters,  the  other  half  were  used  for  filter  testing.  Filter  perfor¬ 
mance  for  the  high- amplitude  eye  events  relevant  to  our  project  was  close  to 
100%  dropping  below  90%  for  very  noisy  data  only.  Identifying  saccades  was 
the  task  most  affected  by  noise.  Noisy  data  can  result  from  poorly  adjusted 
eye-tracker  parameters.  Such  adjustments  are  particularly  important  when 
subjects  are  wearing  glasses,  have  droopy  eyelids,  or  narrow  eyes.  Altogether, 
the  filters  performance  was  found  adequate  for  the  purposes  of  our  project. 
The  four  eye-event  filters  (blinks,  saccades,  nystagmus,  and  pupillary  reflex) 
prototyped  in  MATLAB  in  the  course  of  year  1  were  implemented  in  C++ 
in  year  2.  This  allowed  us  to  increase  the  filters  speed  and  to  unite  stimulus 
generation  and  data  analysis  into  a  feedback  loop  within  the  same  application 
running  on  the  data-analysis  laptop.  The  filters  are  based  on  measuring  and 
thresholding  eye-movement  velocities  and  pupil  area  changes.  From  the  start 
the  filters  were  designed  to  be  used  with  real-time  data:  they  are  fast  and  do 
not  require  large  amounts  of  data.  Less  than  a  second  of  data  is  sufficient  to 
detect  saccades  and  pupillary  reflex,  and  several  seconds  of  data  -  to  detect 
visual  nystagmus  which,  typically,  lasts  for  3-5  seconds  at  a  time.  In  year 
2  we  thoroughly  redesigned  the  nystagmus  and  pupil  dilation  filters  to  make 
their  output  more  robust  to  subjective  variations.  To  this  end  we  developed 
a  new  algorithm,  where  the  eye-tracking  data  is  analyzed  at  different  time 
scales  and  then  the  resulting  filter  outputs  are  integrated  across  the  scales. 
We  also  completely  redesigned  the  filter  used  to  measure  the  orienting  reflex 
(eye-movement  and  the  following  viewing  of  an  interesting  object).  Instead 
of  relying  on  the  direction  of  the  first  saccade  we  used  a  different  approach 
based  on  measuring  how  long  the  subject’s  gaze  stayed  within  the  area  of 
interest  compared  to  the  rest  of  the  scene.  This  measure  proved  to  be  more 
robust  to  the  effect  of  the  (somewhat  unpredictable)  gaze  position  at  the 
beginning  of  the  trial. 

Figure  5  illustrates  the  filters  performance.  Raw  eye-tracking  data  (a 
single  trial  in  each  case)  are  shown  by  the  black  symbols,  the  outputs  of  the 
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corresponding  filters  are  marked  in  red.  Panel  a  illustrates  a  filter  response 
to  a  significantly  longer  viewing  of  an  odd  (animal  face)  image  shown  in  the 
bottom-right  quadrant.  Panel  b  illustrates  a  similar  paradigm,  where  a  semi¬ 
nude  female  image  shown  in  the  left  half  of  the  screen  was  viewed  longer 
than  a  ’boring’  male  image  in  the  right  half.  The  filter  response,  again, 
indicates  a  significantly  longer  viewing  of  the  ’interesting’  image.  Panel  c 
illustrates  a  robust  visual  nystagmus  arising  in  response  to  the  stimulus  where 
a  coherent  drift  of  some  dots  is  masked  by  incoherent  random  movements  of 
the  remaining  dots.  Panel  d  illustrates  a  pupillary  dilation  in  response  to  a 
mild  curse  word  suddenly  appearing  within  a  sequence  of  ’ordinary’  words 
presented  on  the  screen. 

The  visual  stimuli  and  the  respective  filters  were  optimized  in  year  3  to 
make  the  filter  performance  maximally  independent  of  subjective  variations 
in  the  eye-tracking  data.  This  is  most  crucial  while  detecting  the  pupillary 
response  to  a  surprising  stimulus  (e.g.,  a  curse  word),  where  the  latency, 
amplitude,  and  duration  of  the  pupil  dilation  were  found  to  vary  significantly 
from  trial  to  trial  and  also  among  subjects. 

Testing  normal  observers 

Once  the  Northeastern  University  IRB  and  MEDCOM  USAMRMC  approvals 
for  work  with  human  subjects  were  renewed  in  October  2011  we  collected  and 
analyzed  data  for  twenty  five  normal  observers  and  two  observers  with  pos¬ 
sible  visual  dysfunctions.  120  more  observers  were  tested  in  the  course  of 
year  3.  The  large  number  of  observers  is  necessary  to  accumulate  enough 
statistics  to  estimate  the  distribution  of  test  scores  (percent  correct)  among 
normal  observers.  This  distribution  was  later  used  to  convert  the  test  scores 
to  the  probability  of  visual  dysfunctions. 

Northeastern  University  undergraduate  students  of  both  sexes  18  -  25 
years  of  age  with  normal  or  corrected  vision  were  recruited  as  normal  sub¬ 
jects.  The  subjects  were  inexperienced  observers  naive  to  the  purpose  of  the 
test.  The  only  instructions  received  from  the  experimenter  were  to  attend 
to  the  screen.  Before  the  test  all  subjects  were  questioned  on  any  visual 
dysfunctions  they  might  have  had  and  only  those  subjects  reporting  normal 
vision  were  used.  The  subjects  and  experimental  conditions  were  meant  to 
simulate  testing  cognitively  impaired  soldiers.  Experimental  sessions  lasted 
6  minutes  during  which  80  visual  stimuli  were  shown.  Subjects  were  viewing 
stimuli  on  a  21”  LCD  monitor  from  a  distance  of  60  -  80  cm.  The  head- 
mounted  version  of  the  eye-tracker  was  used  for  all  subjects,  pupil  (left  eye) 
position  on  the  camera’s  CCD  was  sampled  at  120  Hz.  The  data  were  col¬ 
lected  and  analyzed  by  the  developed  software  filters  in  real  time  and  then 
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saved  on  the  hard  drive  of  the  data-analysis  laptop.  A  fragment  of  the  data 
and  the  accompanying  automatic  eye-event  segmentation  for  one  subject  are 
shown  in  Figure  5. 

Testing  observers  with  visual  dysfunctions 

Dne  to  the  termination  of  the  Pi’s  contract  with  Northeastern  University 
in  July  2013,  we  were  unable  to  perform  measurements  on  a  large  cohort 
of  patients  with  visual /cognitive  dysfunctions,  which  were  planned  for  the 
terminal  stages  of  the  project.  However,  in  the  course  of  the  project  we  were 
able  to  identify  and  test  four  subjects  with  recent  history  of  brain  injury 
or  visual  dysfunctions.  Given  the  small  number  of  observers  with  visual 
dysfunctions  tested,  it  is  impossible  to  draw  definite  conclusions  about  the 
effectiveness  of  our  approach  to  testing  visual  dysfunctions  using  passive 
oculomotor  reflexes.  However,  the  results  presented  below  make  a  case  for 
cautious  optimism. 

Subject  SI  (a  female,  37  years  old)  had  a  severe  traumatic  brain  injury 
8  months  prior  to  the  test.  As  a  result,  her  control  of  eye  movements  was 
initially  disrupted  to  the  extent  that  for  several  months  she  could  not  read 
at  all,  could  not  do  her  work  or  travel.  At  the  time  of  the  test  she  described 
her  visual  functions  as  almost  normal.  Superficially,  her  eye  movements  also 
appeared  normal.  However,  compared  to  normal  subjects,  SI  showed  very 
low  scores  in  the  coherent  dot  motion  test:  her  performance  fell  below  that 
for  92%  of  normal  subjects.  This  is  even  more  surprising  given  that  SI,  being 
a  vision  scientist  herself,  was  exposed  to  similar  stimuli  more  frequently  than 
the  tested  normal  observers.  Such  low  probability  score  strongly  suggests 
that  Si’s  complex  motion  processing  was  not  restored  to  its  normal. 

Subject  S2  (a  male,  70  years  old)  had  a  foveal  retinal  detachment  in  both 
eyes.  A  year  prior  to  the  test  he  was  operated.  The  first  operation  was 
unsuccessful  in  the  right  eye,  but  after  the  second  operation  he  reported  his 
vision  to  be  mostly  restored  in  both  eyes.  He  was  wearing  his  normal  vision 
correction  during  the  experiment.  Subject  S2  showed  very  low  scores  for 
contrast-defined  nystagmus:  for  the  stimulus  requiring  the  highest  acuity  his 
performance  was  lower  than  that  for  91%  of  normal  subjects.  In  addition, 
his  performance  in  the  color  nystagmus  task  lower  than  for  82%  of  normal 
subjects.  These  results  indicate  that  S2’s  foveal  vision  was  relatively  poor, 
lacking  in  color  sensitivity  and  acuity. 

Subjects  S3  (a  male,  21  years  old)  and  S4  (a  male,  20  years  old)  were 
picked  from  the  general  pool  of  normal  observers  based  on  their  own  verbal  re¬ 
port  of  suspected  color  vision  deficiencies.  Both  subjects  took  an  online  color- 
deficiency  test  (http://www.biyee.net/color-science/color-vision-test)  imrne- 
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diately  prior  to  the  experiment.  The  test  confirmed  that  the  subjects  had 
significant  green  blindness:  their  percentages  of  correct  responses  on  the 
deuteranopia  test  were  well  below  the  corresponding  threshold.  Both  subjects 
reported  that  they  could  see  the  drifting  grating  most  of  the  time,  however, 
their  performance  in  this  test  was  lower  than  for  90%  of  normal  subjects. 
This  indicates  that  the  red  and  green  colors  used  for  our  color-defined  grat¬ 
ing  were  perceived  by  these  subjects  not  equiluminant,  apparently,  due  to 
their  green  cone  deficiencies. 

Data  Analysis 

In  the  second  half  of  year  3  we  analyzed  ocular  responses  of  120  normal 
subjects  recorded  in  the  previous  phases  of  the  project.  The  large  number 
of  normal  observers  was  necessary  to  accumulate  enough  statistics  to  esti¬ 
mate  the  distribution  of  normal  test  scores  (percent  correct).  The  calculated 
normative  distributions  are  plotted  in  the  left  panel  of  Figure  6.  Blue  bars 
represent  the  probability  distribution  of  evoking  and  detecting  an  ocular  re¬ 
flex  for  a  particular  stimulus  as  specified  by  the  title  of  each  subplot.  Red 
bars  represent  the  same  for  a  ’Null’  response,  which  simulates  the  break¬ 
down  of  normal  responses  for  patients  with  severe  visual  dysfunctions.  The 
Null  distributions  were  calculated  by  using  eye  movements  evoked  by  all  the 
stimuli  dissimilar  to  the  tested  one,  i.e.,  irrelevant  to  the  tested  stimulus. 
For  example,  for  a  condition  in  which  visual  nystagmus  was  tested  all  the 
stimuli  except  the  ones  with  drifting  gratings  were  used  to  calculate  the  Null 
distribution.  The  resulting  predictive  power  of  the  classification  between  a 
normal  and  a  dysfunctional  observer  is  proportional  to  the  extent  the  two 
distributions  (blue  and  red)  are  separated.  The  receiver  operating  character¬ 
istic  (ROC)  analysis  was  performed  to  measure  the  power,  and  the  resulting 
ROC  curves  are  shown  in  the  right  panel.  The  area  under  each  ROC  curve 
gives  the  probability  of  a  normal  subject  performing  better  than  a  severely 
dysfunctional  subject  in  the  particular  task.  The  corresponding  probability 
is  shown  by  the  number  within  each  subplot.  The  results  allowed  to  iden¬ 
tify  those  tests  which  have  high  predictive  power  and  those  test  which  do 
not.  For  example,  the  orienting  reflex  to  an  animal  face  among  human  faces 
appears  to  be  very  robust,  whereas  the  orienting  reflex  to  an  upside-down 
human  face  among  normal-side-up  human  faces  is  so  weak  that  it  has  almost 
zero  predictive  power. 

Taking  75%  probability  as  the  threshold  test  power  we  see  that  most  of 
the  stimuli  used  in  the  tests  can  be  used  for  testing  observers  in  a  passive 
way,  i.e.  without  the  observers  being  aware  of  any  task.  Thus,  all  nystagmus 
stimuli  except  for  the  coherent  drifting  dots  are  acceptable.  The  latter  one 
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is  close  to  the  threshold  (71%),  and  can  also  be  used  once  the  dot  coherence 
is  increased  above  the  currently  used  25%.  These  stimuli  can  be  used  to  test 
visual  acuity  (coarse,  medium,  fine  gratings),  color  deficits  (color  gratings) 
and  complex  motion  processing  deficits  (coherent  dots  drift).  The  two  of  the 
orienting  reflex  stimuli  passed  the  power  threshold:  simple  target  (disk)  ori¬ 
enting,  and  orienting  to  complex  objects  (animal  face  among  human  faces). 
These  stimuli  can  be  used  to  test  dysfunctions  at  early  and  late  stages  of 
visual  processing.  Finally,  performance  for  the  stimuli  used  for  testing  visual 
processing  of  text  (65%  and  62%)  did  not  pass  the  75%  threshold.  Careful 
investigation  of  raw  eye-tracking  data  showed  that  the  low  performance  re¬ 
sulted  from  spurious  pupil  dilations  irrelevant  to  the  surprise  words  used  in 
the  test.  Most  likely,  these  dilations  were  caused  by  subjects  shifting  their 
gaze  back  and  forth  between  the  stimulus  text  shown  in  black  on  bright 
white  field  and  the  surrounding  screen  areas  colored  in  neutral  gray.  These 
eye  movements  could  cause  abrupt  changes  in  average  illumination  of  the 
fovea,  which  would  result  in  the  observed  spurious  pupil  dilations  and  con¬ 
tractions.  We  surmise  that  the  spurious  pupil  reactions  can  be  lessened  or 
completely  avoided  by  using  text  nearly  equiluminant  to  the  neutral  gray 
background,  e.g.,  shown  in  blue  matched  in  luminance  to  the  background. 
We  find  it  likely  that  stimulus  modifications  along  these  lines  could  raise  the 
performance  above  the  threshold  level. 
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Figure  5:  Eye  tracking  data  and  the  corresponding  filter  outputs.  Time  from  the 
stimulus  onset  is  plotted  along  the  x-axis  in  panels  b  -  d.  (a)  An  animal  face 
was  shown  in  the  bottom- right  quadrant,  3  human  male  faces  were  shown  in  the 
remaining  quadrants.  Gaze  point  readings  (x  and  y  coordinates  on  the  camera’s 
CCD)  are  marked  by  the  black  dots.  Only  3  images  were  explored  by  the  subject’s 
gaze.  The  red  circle  indicates  a  significantly  longer  viewing  of  the  bottom-right 
image  detected  by  the  filter,  (b)  a  semi- nude  Marilyn  Monroe  image  was  shown  in 
the  left  half  of  the  screen,  a  male  figure  in  a  suit  was  shown  in  the  right  half.  Gaze 
horizontal  position  is  plotted  along  the  y-axis.  A  significantly  longer  viewing  of 
the  left  image  detected  by  the  filter  is  indicated  by  the  red  arrow,  (c)  For  the  first 
quarter  of  the  trial  randomly  placed  dots  were  moving  in  a  ’random  walk’  fashion, 
afterwards  25%  of  the  dots  were  also  drifting  to  the  right.  Gaze  horizontal  position 
is  plotted  along  the  y-axis.  The  red  bar  indicates  a  visual  nystagmus  event  detected 
by  the  filter,  (d)  A  mild  curse  word  was  displayed  on  a  white  background  at  the 
beginning  of  the  trial.  Pupil  area  is  plotted  along  the  y-axis.  The  red  bar  indicates 
pupil  dilation  detected  by  the  filter.  The  pupil  contraction  at  the  beginning  of  the 
trial  resulted  from  the  response  to  the  white  background  of  the  text. 
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Figure  6:  Left  panel:  Normative  response  distributions  based  on  results  for  120 
observers.  Blue  bars  show  distributions  for  normal  observers,  red  bars  show  Null 
distributions  modeled  on  task-irrelevant  eye  movements  of  the  observers.  Right 
panel:  receiver  operating  characteristic  (ROC)  analysis  of  the  predictive  power 
for  each  task.  Number  within  each  subplot  indicates  the  probability  of  correctly 
detecting  a  dysfunction  for  the  tested  reflex  assuming  a  complete  breakdown  of 
the  reflex  for  each  task. 
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Key  Research  Accomplishments 


•  We  identified  and  purchased  two  eye-tracker  models,  head-mounted  and 
desktop-mounted,  which  can  be  used  to  carry  out  robust  eye-tracking 
for  patients  with  cognitive  disabilities. 

•  A  software  library  for  adaptive  visual  stimulus  generation,  presentation, 
and  data  acquisition  was  developed. 

•  Visual  stimuli  were  designed  and  tested  on  normal  observers  naive  to 
the  purpose  of  the  test.  No  specific  instructions  were  given  to  simulate 
experiments  with  cognitively  impaired  observers.  The  collected  data 
was  used  to  optimize  the  visual  stimulation  and  to  design  software 
filters  which  allow  automatic  detection  of  relevant  reflex  eye  movements 
in  real  time. 

•  Real-time  data  exchange  between  the  eye-tracker  and  the  data-analysis 
computer  was  implemented,  tested,  and  optimized.  This  finalized  the 
hardware  kit  development. 

•  Eye-event  filters  for  automatic  ocular  reflex  detection  were  designed, 
implemented  in  C++,  and  incorporated  into  the  software  running  on 
the  visual  stimulation  and  data  analysis  laptop.  The  real-time  process¬ 
ing  of  the  data  allowed  to  make  the  testing  adaptive. 

•  The  complete  kit  was  tested  on  120  normal  observers  naive  to  the 
purpose  of  the  test  and  four  observers  with  possible  visual  dysfunctions 
either  congenital  or  resulting  from  recent  TBI/eye  injuries. 

•  Using  the  data,  normative  performance  distributions  were  calculated 
for  the  tested  stimuli. 

•  When  compared  with  normal  subjects  the  observers  with  possible  visual 
dysfunctions  showed  significantly  lower  performance  (<  90%  of  the 
norm),  which  appear  to  be  in  agreement  with  their  symptoms  and 
clinical  histories. 
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Reportable  Outcomes 


There  were  no  reportable  outcomes  yet.  We  are  in  the  process  of  preparing 
a  publication  based  on  the  results  of  the  study.  In  particular,  it  will  report 
the  new  result:  unlike  a  salient  object  stimulus,  e.g.,  an  animal  face  among 
human  faces,  a  salient  motion  stimulus,  biological  motion  vs.  scrambled 
motion,  does  not  attract  attention  (does  not  pop-out).  We  also  are  preparing 
to  file  a  patent  describing  the  current  approach  to  passively  assessing  visual 
functions  using  ocular  reflexes. 
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Conclusions 


We  successfully  developed  and  implemented  a  battery  of  visual  tests  de¬ 
signed  to  diagnose  various  visual  dysfunctions  by  means  of  recording  and 
objectively  analyzing  passive  oculomotor  reflexes.  All  hardware  and  software 
components  of  the  data  acquisition  and  analysis  pipeline  were  finalized  and 
tested  in  realistic  experimental  conditions  on  normal  observers  and  several 
observers  with  visual  dysfunctions.  Observers  received  no  instructions  from 
us  regarding  their  task  except  for  attending  to  the  screen.  This  passive  view¬ 
ing  regime  imitated  experimental  conditions  for  testing  cognitively  impaired 
patients.  The  project  results  indicate  that  the  adopted  paradigm  has  suffi¬ 
cient  statistical  power  to  successfully  diagnose  various  visual  dysfunctions, 
which  validates  the  proposed  approach. 
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